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Abstract. The relation between agentive action, knowledge, and obligation is 
central to the understanding of responsibility -a main topic in Artificial Intelli¬ 
gence. Based on the view that an appropriate formalization of said relation would 
contribute to the development of ethical AI, we point out the main characteris¬ 
tics of a logic for objective and subjective oughts that was recently introduced 
in the literature. This logic extends the traditional stit paradigm with deontic and 
epistemic operators, and provides a semantics that deals with Horty’s puzzles for 
knowledge and obligation. We provide an axiomatization for this logic, and ad¬ 
dress its soundness and completeness with respect to a class of relevant models. 


1 Introduction 

AI developers face a big challenge in creating systems that are expected to make ethi¬ 
cally charged decisions. The field of machine ethics has seen a quick growth in recent 
years, and questions regarding responsibility of autonomous agents are very important. 
In our opinion, these questions can be categorized in two main trends: 1) conceptual 
questions that deal with the ontology and essential components of the notion of respon¬ 
sibility, and 2) technical questions concerning the implementation of such notion in AI. 
This work attempts to make a contribution in the technical category. We take part in a 
very specific line of research, where proof systems of deontic logic are intended to help 
in the testing of ethical behavior of AI through theorem proving ([1], [9], [3]). That 
being said, answers to the technical kind of questions typically presuppose a particu¬ 
lar choice of philosophical standpoint against questions of the first kind, the conceptual 
one. Our philosophical standpoint comes from John Horty’s framework of act utilitarian 
stit logic ([8]), extended with epistemic relations. As such, we make explicit the goal of 
the present paper: to provide well-behaved formalizations of 3 essential components of 
responsibility of intelligent systems: agentive action, knowledge, and obligation. 

According to [1], having an expressive deontic logic with practical relevance and 
an efficient algorithm for proving theorem-hood is highly applicable in the construction 
of logic-based ethical robots.^ To support this statement, Arkoudas et al. present in [1] 
a natural deduction calculus for a logic of ought-to-do that was developed by Horty 
in [8] and axiomatised by Murakami in [9]. With an interactive theorem proving sys¬ 
tem named Athena, they illustrate the fact that we can mechanize deontic logic to do 
machine ethics. 

^ See [10] for an overview of the advantages and disadvantages of doing machine ethics via 
theorem proving. 



The starting point of our work comes from a recent interest in enhancing both the 
expressivity and practical relevance of Horty’s stit theory of ought-to-do in order to deal 
with situations in which agents’ knowledge becomes significant. Inspired by 3 puzzles 
for knowledge-dependent obligations that pose a problem for merely extending his ini¬ 
tial logic with epistemic operators, Horty presents in [6] a novel semantics for epistemic 
oughts based on action types. Although the approach is substantial, it comes with 3 dis¬ 
advantages: (1) it diverges from his work in [8], (2) there are semantic constraints that 
limit the expressivity of the models, and (3) the use of action types precludes an efficient 
axiomatization."* 

With similar motivations as Horty, the extended abstract [5] proposes an alternative 
logic -where the main idea is to distinguish objective from subjective obligations- that 
allegedly mitigates the disadvantages mentioned above. The authors claim that this logic 
is simpler, more naturally connected to Horty’s work in [8], and axiomatizable. Being 
very brief, [5] only deals with the conceptual benefits of the proposal, and the proof of 
soundness and completeness of the logic’s proof system is only mentioned. Here, we 
recover the definition of this logic, address its benefits, and show that its proof system is 
indeed sound and complete with respect to a class of relevant models. We do this hoping 
that the results will give some background to new developments in the mechanization 
of deontic logic for ethical AI, in the aforementioned tradition of [1] and [3]. 

The paper is structured as follows. After a short presentation of stit and its apli- 
cations in the modeling of action, knowledge, and obligation, we go through Horty’s 
puzzles. We mention his solution to them and justify the claim that his approach with 
action types comes with disadvantages. Afterwards, we present the logic developed by 
[5], show how it solves Horty’s puzzles, and deal with its axiomatizability.^ 

2 Action, Knowledge, and Obligation in Stit 

We want to consider oughts from the perspective that an agent should be excused for 
having failed at an obligation if it lacks the necessary knowledge to perform a required 
task (doctors ought to stop the bleeding of the patient, but if they do not know how to, 
they should be excused). A typical framework for expressing statements that involve 
knowledge for required tasks as a component of responsibility is stit logic ([2]). Stit 
logic was created to formalize the concept of action, so it naturally lent itself to the study 
of obligation ([8]) and of knowingly doing ([4]), all important elements in the notion of 
responsibility. For a comprehensive review of the interaction of these 3 concepts in the 
literature, we refer the reader to [11]. 

The 3 puzzles of Horty that we mentioned in the introduction are actually very good 
illustrations of how stit deals with action, knowledge, and obligation, but in order to 
tackle them we need to recover basic definitions of the paradigm known as act utilitar¬ 
ian stit logic. In this paradigm, obligation stems from a dominance ordering over the set 
of available actions. The idea is that the effects of the best actions in the ordering -the 

We will justify all these claims in the second section of this paper, after introducing the 3 
puzzles and addressing Horty’s solution. 

^ The proof of completeness is dense and technical, so the full proofs of each statement are 
provided in the appendix section of the present work. 



so-called optimal actions- are the obligations of a given agent at a given moment. We 
proceed to introduce the basic aspects of an extension of act utilitarian stit logic with 
epistemic modalities, and leave its examples to the section where we present Horty’s 
puzzles. 

Definition 1 (Syntax). Given a finite set Ags of agent names, a countable set of propo¬ 
sitions P such that p € P and a € Ags, the grammar for the formal language Cko 
given by: 

ip := p \ ^ip \ ip Alp \Dip \ [a\p \ K^p \ Q[a\p 

□(/? is meant to express the ‘historical necessity’ of p {()p abbreviates [c(\p 

stands for ‘agent a sees to it that p\ is the epistemic operator for a. Finally, ©[q;]v 3 
is meant to represent that a ought to see to it that p. 

As for the semantics, the structures in which we evaluate formulas of the language 
>Cko ^6 based on what we call epistemic act utilitarian branching time frames. 

Definition 2 (Branching time (BT) frames). A finite-choice epistemic act utilitarian 
BT-frame is a tuple (T, □, Choice, Value) such that: 

- T is a non-empty set of moments and \Z is a strict partial ordering on T satisfying 
‘no backward branching’. Each maximal \Z-chain is called a history, which repre¬ 
sents a way in which time might evolve. PI denotes the set of all histories, and for 
each m £ T, Hm '.= {h & H;m G h}. Tuples {m,h) are called situations ijf 
m G T, h G H, and m £ h. Choice is a function that maps each agent a and 
moment mto a finite partition Choice™ of Hm, where the cells of such a partition 
represent a’s available actions at m. Choice satisfies two constraints: 

• (NC) or ‘no choice between undivided histories’.- For all h, h' € Hm, if m' G 
h n h' for some m' □ m, then h € L iffh' G L for every L G Choice™. 

• (lA) or ‘independence of agency’.- A function s on Ags is called a selection 

function at m if it assigns to each a a member o/Choice™. If we denote by 
Select™ the set of all selection functions at m, then we have that for every 
m € T and s G Select™, HaeAgs ^ ® discussion of the 

property). 

- For a G Ags, ~q, is the epistemic indistinguishability equivalence relation for 
agent a. 

- Value is a deontic function that assigns to each history h G H a real number, 
representing the utility ofh. 

As mentioned before, the idea is that obligations come from the optimal actions for 
a given agent. The optimality of such actions is relative to a dominance ordering of the 
actions, and this ordering is given by the value of the histories in those actions -itself 
provided by Value. In order to present the semantics for formulas involving the ought- 
to-do operator, we therefore need some previous definitions. For a G Ags and m,, G T, 
we define a dominance ordering ^ on Choice™* such that for L, L' G Choice™*, 
L f L' iff Value (ft,) < Value (ft') for every ft G L,h' G L'. We write L P L' iff 
PPL' and L' ^ L. The optimal set of actions, then, is taken as Optimal™* := 
{L G Choice™*; there is no P' G Choice™*such that L P L'}. As is customary, the 
models and the semantics for the formulas are defined by adding a valuation function 
to the frames of Definition 2: 



Definition 3. A BT-model M. consists of the tuple that results from adding a valuation 
function V to a BT-frame, where V : P —>■ 2^^^ assigns to each atomic proposition 
a set of moment-history pairs. Relative to a model A4, the semantics for the formulas 
of Cko defined recursively by the following truth conditions, evaluated at a given 
situation {m, h): 
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Satisfiability, validity on a frame, and general validity are defined as usual. We write 
\ip\'^ to refer to the set {h G Hm] A4, (to, h) |= ip}. 


2.1 Horty’s Pnzzles 

The 3 puzzles that we have mentioned since the beginning of the paper, and that pose 
a problem for formalizing epistemic oughts just with the epistemic extension of act 
utilitarian logic, can be summarized as follows. 

Example 1. Agent places a coin on top of a table -either heads up or tails up- but 
hides it from agent a. Agent a can bet that the coin is heads up, that it is tails up, or 
it can refrain from betting. If a bets and chooses correctly, it wins €10. If it chooses 
incorrectly, it does not win anything, and if it refrains from betting, it wins €5. 


hi /i2 /i3 h.! hs he 



Fig.l : Coin problem #1 


The stit diagram that represents Horty’s interpretation of the situation is included 
in Figure 1. In this diagram, we take H to denote the proposition ‘/? places the coin 
heads up’, T to denote ‘/3 places the coin tails up’, BH to denote ‘a bets heads’, BT to 
denote ‘a bets tails’, and G to denote ‘a gambles’. In moment toi, /3 places the coin on 











top of the table, so that its available actions are labeled by Li (placing the coin heads 
up) and L 2 (placing the coin tails up). At moments m 2 and m 3 , it is a’s turn to act, 
and the available actions are clear from the picture. The blue dotted line represents the 
epistemic class of a: since jS is hiding the coin, a cannot distinguish whether it is at 
moment m 2 or m 3 .® For such an interpretation regarding the epistemic structure of the 
agent, a problem ensues due to the fact that for every i G {2,3} and h G Hrm, we have 
that {rrii, h) |= Ka © [a\G. This means that a knows that it ought to gamble, even if this 
is a ‘risky’ move that could result in a payoff of 0. In this sense, we could say that the 
agent’s knowledge of what is optimal would lead it into taking a chance and gambling. 

Example 2. With the same scheme as in the previous example, if a bets and chooses 
correctly, it wins €10. If it refrains from betting, it also wins €10. If it bets incorrectly, 
it does not win anything. 

Intuitively, a ought to refrain from gambling in this scenario, for refraining implies that 
it would win by the same amount as when betting correctly but without engaging in an 
action that could possibly fail. In this case, the problem is that for every i G {2, 3} and 
h G Hrui, we have that {mi,h) ^ Ka © [a\-^G\ a does not know that it ought to refrain 
from gambling. Figure 2 includes the stit diagram for this scenario. 


h\ h2 hs h,i /15 he, 



Fig. 2: Coin problem #2 


Example 3. With the same scheme as in the previous examples, if a bets and chooses 
correctly, it wins €10. If it bets incorrectly or refrains from betting, it does not win 
anything. 

The problem here is that for every i G {2,3} and h G Hm,, {mi, h) \= Ka © [a]kF, 
where W is the proposition that stands for ‘a wins’. This means that a knows that it 
ought to win at any given situation, but such knowledge is not action-guiding, meaning 
that it will not provide a with a choice to make. Though the agent knowingly ought 

® Notice that Horty’s formalization also yields that in none of the situations will a knowingly 
perform any of the available actions: it cannot epistemically distinguish between the situations 
in which it is ‘betting heads’, ‘betting tails’, or ‘refraining’. 












to win, it cannot knowingly do so -it simply does not have the means due to a lack of 
knowledge. Thus, Kant’s principle of ‘ought implies can’ is not satisfied {{rrii, h) ^ 
Ka (D [q]W ^ ()Ka[a\W). Figure 3 includes the stit diagram for this scenario. 



L, L2 


Choice] 


Fig. 3: Coin problem #3 


Horty solves these 3 puzzles by introducing both syntactic and semantic addenda to 
epistemic act utilitarian stit logic. He extends the language with an operator [... kstit] 
to encode the concept of ex interim knowledge (or knowingly doing). The semantics for 
formulas involving this operator uses action types, with the premise that actions of the 
same type may lead to different outcomes in different moments (see [7]). Unfortunately, 
the introduction of the new operator and of types comes with two unfavorable semantic 
constraints: 

1. In order for [... kstit] to be an S5 operator, the epistemic relations must ensue not 
between moment-history pairs but between moments. The problem with this con¬ 
dition is that it limits the class of models to those in which knowledge is moment- 
dependent (agents will not be able to know that they perform a given action),^ 

2. Indistinguishable moments must offer the same available types. The problem with 
this constraint is that it cannot be characterized syntactically without producing an 
infinite axiomatization. This is due to the fact that performing a certain action type 
can only be expressed syntactically with propositional constants (again, see [7] for 
the details).^ 

As a solution to the stated puzzles, Horty’s approach is successful. However, [5] 
claims that we can also be successful without using action types. The benefits of the 
framework presented in [5], then, include technical characterizability of the constraints 
imposed on the structures -which is important for axiomatization-, semantic simplicity, 
and enhanced expressivity. In the following section, all these claims will be substanti¬ 
ated. 

^ Horty’s models satisfy the following constraint: if {m,h) {m!,h') then {m,h') 

(m'j h”) for every h' € 77m, h” £ 77m' • 

* It is an open problem to determine whether there is a finite axiomatization of Horty’s logic of 
epistemic action and obligation if the types were also included in the object language. 












3 A Logic of Objective and Subjective oughts 


[5] proposes to disambiguate two senses of ought-to-do in order to produce a system 
that solves Horty’s puzzles, avoids action types, and is axiomatizable. The two senses 
are an objective one, which coincides with Horty’s act utilitarian ought-to-do, and a 
subjective one, which arises from the epistemically best candidates in the set of avail¬ 
able actions for a given agent. By ‘epistemically best’ we mean those actions that are 
undominated not only in the actual moment but whose all epistemic equivalents across 
different indistinguishable situations are also undominated. 

Essentially, we are talking about an extension of the language in Definition 1 with a 
new operator ©5 [a], meant to build up formulas that would express what a subjectively 
ought to do. As for the semantics of this new operator, it involves a dominance ordering 
as well, but one different to that which is used for objective ought-to-do’s. In order 
to define this subjective dominance ordering, [5] introduces a new semantic concept 
known as epistemic clusters, which are nothing more than a given action’s epistemic 
equivalents in situations that are indistinguishable to the actual one. Formally, we have 
that for a € Ags, m* ,m G T, and L C , L’s epistemic cluster at m is the set 

[L]™ := {h e Hjn] e L s.t. (m*, h*) {m, h)}. 

As a convention, we write m m' if there exist h G H^, h' € iTm' such that 
{m,h) ~Q, {m!,h'). The notion of epistemic clusters is used to define a subjective 
dominance ordering on Choice™* by the following rule: for L, L' G Choice™*, 
L ©g L' iff for every m such that m* ~q, to, Value(h) < Value(h') for every 
h G [L]™,/i' G [i^]™- Just as in the case of objective ought-to-do’s, this ordering 
allows us to define a subjectively optimal set of actions S — optimal™* := {L G 
Choice™*; there is no L' G Choice™* s. t. L ©g L'}, where we write L ©g L' iff 
L ©g L' and L' ^g L. The idea, then, is that something will be a subjective obligation 
of a given agent at a given moment if it is an effect of all the subjectively optimal actions 
of that agent at that moment. 

As established in [5], the models in which to evaluate the formulas of the extended 
language need to satisfy extra constraints in order to capture an appropriate interaction 
of action, knowledge, and subjective obligation. By this we mean to say that in these 
models (a) agents should be able to knowingly do the same things across epistemically 
indistinguishable states, (b) the subjective ought-to-do must conform to Kant’s direc¬ 
tive of ‘ought implies can’ in its epistemic version of ‘subjectively ought-to-do implies 
ability of knowingly doing’, and (c) if something is a subjective ought-to-do of a given 
agent, then the agent should know that that is the case. Therefore, we focus on models 
that fulfill the following requirements, which will grant the conditions mentioned above 
(as can be seen from the proof of soundness): 

- (OAC) For every situation (to*, ft.*), if (to*, ft*) (w, ft) for some (to, ft), then 
(to*, ft),) (to, ft) for every ft) G Choice™* (ft*). We refer to this constraint as 
the ‘own action condition’, since it implies that agents do not know more than what 
they perform. Because of this constraint, the knowledge that we are formalizing 
here is of a very particular kind: to know something is just the same as to knowingly 
do it. 



- (Unif — H) For every situation (m*, /i*), if (m*, /i*) (m, /i) for some (to, /i), 

then for every h'^ G Hm,, there exists h' G such that (to*, /i*) (to, /i'). 
Combined with (OAC), this constraint is meant to capture a notion of uniformity 
of strategies, where epistemically indistinguishable situations should offer similar 
actions for the agent to choose upon. We call this condition ‘uniformity of historical 
possibility’. 

In finite-choice epistemic act utilitarian BT-models that satisfy these two constraints, 
then, the semantics for the formulas involving ©5 [a] is defined as expected: 

(to,/ i) ^ 05[a](p iff VL S S — optimal™, Vto' s.t. TO ~ct [L]'^ C |(p|™ . 

3.1 Solution to Horty’s Puzzles 

The semantics for subjective ought-to-do’s offers solutions to natural interpretations 
of Horty’s puzzles, in which the assumption that the coin is hidden from the bet¬ 
ting agent is captured by taking to be defined by the following information sets: 
{{m2, hi), (to 3 ,/i 4 )}, in which a bets heads; {(to 2 ,/i 2 )j (w 3 ,/i 5 )}, in which a bets 
tails; and {(to 2 , ft. 3 ), (tos, hg)}, in which a refrains from betting. 

For Example 1, the problem is solved because although (to^, hi) ^ K^, © [a]G, 
we consider this as the knowledge of an objective ought-to-do. Subjectively speak¬ 
ing, we do not obtain that a knows that it ought to gamble: (to^, hi) ^ ©^[aJG and 
thus {mi, hi) ^ Ka ©5 [a]G. This can be seen by noticing that S — Optimal™^ = 
{L 3 , Li, L 5 } and S — Optimal™^ = {Lq, L^, Lg}. Figure 4 provides a stit diagram 
for this scenario. 



Li Lj 


Choice’. 


Fig. 4: Coin problem #1, revisited 


In Example 2, the problem is solved because {mi, hi) \= ©^[aJ^G and {mi, hi) ^ 
Ka Os [ctJ^G (notice that S — Optimal™^ = {L 5 } and S — Optimal™^ = {Eg}). 

Eor Example 3, the problem is solved because we obtain that {mi, hi) ^ ©5 [a]IE, 
which in turn implies that {mi, hi) ^ Os [c(]W (notice that S — Optimal™^ = 
{L3, Li, L^} and S — Optimal™^ = {Lq, L^, Ls}). Therefore, although a knows 
that it objectively ought to win, it is not the case that it subjectively ought to win. 












When comparing these solutions to Horty’s, it is important to point out that the 
formalization we are using is different from his, for it presupposes that the indistin- 
guishability relation for agent a ensues between situations. Regardless of such dif¬ 
ference, the solution is virtually the same: for Example 1, Horty gets that for every 
h G Hm 2 (w 2 , h) ^ ©[a kstitJG. For Example 2, he gets that for every h G Hm^ 
(m 2 ,/i) \= ©[a kstitJ^G, and for Example 3, he gets that for every h G -ffmj 
(m 2 , h) ^ ©[a kstitJfE. Therefore, we can see that the notion of © 5 [a] works as 
an analog of ©[... kstit]. 

4 Axiomatization and Some Logical Properties 

In this section, we introduce a proof system for the logic presented, address its sound¬ 
ness and completeness results, and mention a few interesting properties of it. 

Definition 4 (Proof system). Let A be the proof system defined by the following axioms 
and rules of inference: 

(Axioms) 


- All classical tautologies from propositional logic. 

- The S5 axiom schemata for □, [a], Kaf 

- The following axiom schemata for the interactions of formulas with the given oper- 


ators: 


© [a](p ->• q) (0[a]p ->• G)[a]q) 

(Al) 

□p —>■ [a]p A Q[a]p 

{A2) 

□ 0 [cx\p V □-! © [a]p 

(Ai) 

© [a\p ->• ©[a]([a]p) 

(AL) 

© [a]p ->• 0[«1p 

For n > 1 and pairwise different ai,..., an, 

/ \ 

(Oic) 

(\ ()[ai]pi 0 1 /\ [oti]pi j 

(lA) 

l<fe<n \ l<fc<n J 


Qs [a\(p -)■ (?) -)■ (©sHp -)■ ©5 [«]<?) 

(Ab) 

Qs [a]p Qs[a]{Kap) 

{A&) 

KaP [a]p 

(OAC) 

OKaP -)■ KaOp 

(Unif - H) 

KaOp ->• ©sHp 

(s.N) 

Qs [a]p ->■ OffaP 

(s.Oic) 

©5 [a\p ©5 [oi]p 

(Cl) 


(Rules of inference) 

- Modus Ponens, Substitution, and Necessitation for the modal operators. 

* The S5 axiom schemata are standard from modal logic, and we include their names here just 
for coherence. They are (K), (T), (4), and (5). 



Schema {IA) encodes ‘independence of agency’. (OAC) encodes the ‘own action 
condition’. {Unif — H) encodes the ‘uniformity of historical possibility’ constraint. 
{Oic) and (s.Oic) concern the objective, resp. subjective, versions of Kant’s directive of 
‘ought implies can’. (s.N) (standing for ‘subjective necessity’) captures that all that is 
historically necessary at epistemically indistinguishable situations must be a subjective 
obligation, and (Cl) (standing for ‘closure’) characterizes a property that says that if 
one subjectively ought to do something, then one knows that that is settled. 

The axiom system A turns out to be sound and complete with respect to the class 
of epistemic act utilitarian bi-valued BT-models. Such models are more general than 
the ones introduced in Definition 3. Instead of only one value function, there are two: 
one for the objective ought-to-do’s, and the other for the subjective ones.'® These mod¬ 
els, then, are of the form (T, □, Choice, {^ajaeAgs, Valueo, Valuer). As such, the 
models in Definition 3 are particular instances of bi-valued models, in which both value 
functions assign the same value to each history of the tree. 

Furthermore, our soundness and completeness results presuppose a logic of ought- 
to-do that deals with the ‘sure-thing principle’, according to which the ranking of the 
available actions of a given agent should take into consideration what all the other 
agents choose concurrently. For a given agent, the action profiles of the other agents 
are seen as the possible states in which the agent will act (see [8], Chapter 4, subsection 
4.1.2). We implement Horty’s approach for dealing with the ‘sure-thing principle’ in 
the logic that we axiomatize, and this means that the semantics that we use is in fact a 
generalization of the one introduced before. In order to address such a generalization 
and the soundness and completeness results, we need further definitions: 

For m G T and (3 G Ags, we define 

State™ = {S C Hm] S = P| s(a), where s € Select™}. 

aeAgs-{/3} 

With this notion of states, we redefine the dominance orderings so that the actions 
are measured taking into consideration the states which those actions are facing. For 
a G Ags and m* G T, we first define a general ordering < on V{HmS) such that 

This extension is extremely useful for the proof of completeness, but neither its conceptual 
reach nor its philosophical implications has been a subject of our investigation as of yet. There 
are more than a few reasons to entertain skepticism about this technical decision, since it 
is not grounded philosophically. To have two deontic functions instead of one lends itself to 
potentially having different values for histories according to whether their utility is objective or 
subjective. However, we must observe that the notions of objective and subjective utility are not 
what drives the disambiguation of ought-to-do into objective and subjective obligations. What 
drives it is the epistemic structure of a given agent. Therefore, the use of two value functions 
should not be taken as essential for the proposal of the logic presented here. Moreover, the 
soundness result works for models in which there is only one deontic function, and the solution 
to Horty’s puzzles does not rely on there being two. On a related note, we make explicit that 
the focus of this work was to develop well-behaved semantics for subjective ought-to-do’s in 
the stit tradition, so that if we presented a language with only the subjective-ought operator 
and not the objective one, as well as with the operators for knowledge, action, and historical 
necessity, then our proof of completeness would in fact be enough to show completeness of 
this fragment with respect to the class of single-valued epistemic act utilitarian BT-models. 



for X,Y C Hm,, X < Y iff Valuec)(ft,) < Valuec)(/i') for every h G X,h' G Y. 
The objective dominance ordering ^ is now defined such that for L, L' G Choice™*, 
L ^ L' iff VS' G State™*, T n S < L' n S. As for the subjective ought-to-do’s, 
we first define a general ordering <s on such that for X,Y C Hm,, X <s 

Ym Values (h) Ss ’Value,^for all h G X^ h' G Y. The subjective dominance 
ordering is then defined such that for L, L' C Hm,, L L' iff Vm such that 
m* m, VS G State™, [L]™ n S <, [L']™ n S. 

With these definitions, we adapt the semantics for the formulas involving ought-to- 
do operators, making the logic strong enough to deal with the ‘sure-thing principle’: 
we set (to, h) |= ©[ajtp iff for every L G Choice™ such that (to, Hl) for some 
€ L, there exists L' G Choice™ such that L < L' and if L" = L ox L' ^ L", 
then (to, h!) \= ip for every h' G L”. Similarly, we set (to, h) \= iff for every 

L G Choice™ such that (to', h^) Y (p for some G [L]™ (to w'), there exists 
L' G Choice™ such that L L' and if L" = L' ox L' L", then for every to" 

such that TO to", (to", h") 1= (p for every h" G [L"]™”. 

4.1 Soundness 

Proposition 1. The system A is sound with respect to the class of epistemic act utilitar¬ 
ian bi-valued BT-models. 

Proof. Standard. See Appendix for details. 


4.2 Completeness 

For reasons of space, we can only summarize the proof here, leaving the full detailed 
exposition for the Appendix. The proof of completeness of A with respect to the class 
of epistemic act utilitarian bi-valued BT-models is a two-step process. First, we intro¬ 
duce relational structures called Kripke-estit models for evaluating the formulas of the 
language £ko under a semantics that mirrors the semantics of Definition 3, and prove 
completeness of A with respect to these structures via the technique of canonical mod¬ 
els. Secondly, we provide a truth-preserving correspondence between Kripke-estit mod¬ 
els and certain epistemic act utilitarian bi-valued BT-models, so that completeness with 
respect to Kripke-estit models grants it with respect to bi-valued BT-models. Essential 
to the success of the technique of canonical models is to prove the so-called existence 
and truth lemmas just as in modal logic. In the case of this particular proof system, 
these lemmas are quite involved, and the reader is encouraged to go over them in the 
Appendix, which offers enough room to address all their features carefully and with pre¬ 
cision. In our opinion, the results of soundness and completeness are significant, all the 
more because we want to provide some theoretical foundations to the idea -explored by 
[1]- that we can actually do machine ethics via theorem-proving (or model-checking). 

To end on a slightly less technical note, we finish this section by addressing some 
interesting properties concerning interactions of certain operators in the logic presented: 


^ ©[a](/? ()Ka[a]p. 



‘It is not the case that if the agent objectively ought to do something, then it can 
knowingly do it.’ Our solution to Example 3 poses a counterexample, because {m 2 , hi) 
is such that (m 2 , hi) \= ©[a]lE, but (m 2 , hi) ()Ka[c(\W, as witnessed by the facts 
that (m 2 ,/i 2 ) W]W and that (m 3 ,/i 4 ) [a]W. 

‘It is not the case that if the agent subjectively ought to do something, then it objectively 
ought to do it! Our solution to Example 2 poses a counterexample, because (m 2 , hf) is 
such that (m 2 , hi) ^ ©^[aJ^G, but {m 2 , hi) 'f= ©[aJ^G. 

Q[a]ip Qs[c<\‘I^- 

‘It is not the case that if the agent objectively ought to do something, then it subjectively 
ought to do it! Our solution to Example 3 poses a counterexample, because (m 2 , hi) is 
such that (m 2 , hi) \= ©[a]lE, but (m 2 , hi) ©^[aJlE. 

5 Conclusion 

This work deals with important questions in the modeling of agency, knowledge, and 
obligation. Eormal depictions of such concepts are likely to be useful when it comes 
to doing machine ethics based on deontic logic and its mechanization. The approach 
discussed and analyzed here is based on a stit logic of utilitarian ‘ought-to-do’ en¬ 
riched with epistemic relations. We argue that to solve certain problems in the treatment 
of knowledge and obligations within stit -namely Horty’s puzzles- one possibility is 
to distinguish between objective and subjective versions of the ought-to-do modality. 
Moreover, we show that this possibility comes with formal advantages such as simplic¬ 
ity of semantics and axiomatizability. 
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A Proof of Soundness 


Proposition 2. The system A is sound with respect to the class of epistemic utilitarian 
bi-valued BT-models. 

Proof. LetAi = (T, C, Choice, Valueo, Valuer) be an epistemic util¬ 

itarian bi-valued BT-frame. Let V be any pertinent valuation function. The S5 axioms 
for □, [a], Ka, as well as axioms {Al), {A2), (A3), (A4), (A5), (A6), (Oic), {IA), 
(OAC), and {Unif — H) are shown to be semantic validities straightforwardly. Since 
they involve some novelty, we include the detailed proofs for {s.N), (s.Oic), and (Cl) 
here. 

- To see that A4 N (s.N), take such that ^ KaOip. Let m be 

such that TO* ~Q, TO (which means that there exist j G f € Hm such 

that {m^,j) and let L G Choice™*. Condition (Unif — H) ensures 

that there exists ft,'* G Hm such that ~q, {m,h'fj. Our assumption that 

(to*, ft*) 1= KfOitp implies then that (to, ft'*) \= 'Onp. Therefore, for any ft € [L]™, 
the fact that ft € iftm yields that (to, ft) \= (p. Therefore, for every L € Choice™* 
and every to such that to* ~q, to , we have that [L]™ C |(/?|™, which vacuously 
implies that (to*, ft*) |= © 5 [a](p. 

- To see that A4 N (s.Oic), take such that {m^,hf) ^ © 5 [a](/?. This 

implies that there exists L* C Hm, such that [L*]™ C |(/?|™ for every m' G T 
such that TO* w'. Since ~q, is reflexive, it is clear that [L*]™* C |(/?|™*. Now, 
take fto € L*. Let (to, ft) be such that (to*, fto) (tu,, ft). From the definition of 
the epistemic cluster set, we have that ft e [L*]™, so that [L*]™ C \ip\"^ and thus 
(to, ft) \= (fi. Therefore, we have that ftg G Hm, is such that for every (to, ft) with 
(m^,hf) (w, ft), (to, ft) \= ip. This means that {m.^,ho) ^ KaP- Therefore, 

we have that (m„,h^) ^ ()Kap. 

- To see that M N (Cl), take such that (m^,h^) |= © 5 [q;]v 3 . Let (TO,j) 

be such that (w, j). Take ft G Hm. We want to show that for ev¬ 
ery L G Choice™ such that [L]™ % \pY^ (for some to' such that to 

to'), there exists L' G Choice™ such that L -<s L' and if L" = L' or L' <s 
L", then for every to" such that to ~q, to", we have that [L"]™ C \p\^ . Take 
L G Choice™ such that there exists m! G T with to tn' and [ft]™ ^ |(/j|™ . 
Let Nl be an action in Choice™* such that Nl C [L]™*. We know that such 
an action exists, in virtue of (Unif — H) and (OAC). Notice that transitivity of 
entails that [Al]° = [L]° for any moment o, so we have that [iVi]™' 2 ■ 

Since ^ © 5 [q;](^, we get that there must exist N G Choice™* such 

that Nl ©s a and if A' = N ov N N', then for every to" such that to* 
to", [A']™ C \p\'^ . Now, let Lat be an action in Choice™ such that Ln C 
[TV]™ (which implies that = [A]° for any moment o). We claim that L ©^ 

Ljv, and show our claim with the following argument. Let to" be a moment such 
that TO w", and let S G State™ . On one hand, we have that * [L]™ n S' = 
[Al]™” n S <s [iV]™” n S = [Liv]™” On the other hand, we know that 
there exists a moment to'" such that to* to'" and a state So € State™ such 
that [iVj™ n So [Al]™ H Sq. Therefore, we have that (★) [Ltv]™ H Sq = 



[N]f' n ^0 [NL]f n So = [L]-'" n S^. Together, (*) and (**) entail 
that L -<g Ln. Now, let L" G Choice™ such that L" = ov L". If 

L" = Ln, then for every m" such that m ~q, m", [T"]™ = [N]'^ C |i^|™ . 

If Lat -<s L”, then an argument similar to the the one we used above to show that 
our claim was true renders that an action Nl^i G Choice™* with Nlh C [L"]-. 
is such that N -/Vl", so that [T"]™ = [-/V^//]™ C \ip\’^ . With this, we have 
shown that (m, h) ^ ©5[q;](/?. Since this happens for every h G Hm, we have that 
{m,j) 1= □ ©5 \a](p. Since this happens for any {m,j) such that 
{m,j), then we get that ^ KaH ©5 [a]ip. 

- Modus Ponens, Substitution, and Necessitation for □ preserve validity. 

Therefore, we have shown that the system A is sound with respect to the class of epis- 
temic bi-valued BT-models. 

B Proof of Completeness 

Definition 5 (Kripke-estit frames). 

A tuple T = (IT, Choice, {sSctjagAgg, Valueo, Valuer) is called a Kripke- 
estit frame iffW is a set of possible worlds and 

1. R\j is an equivalence relation over W. For w G IT, the class of w under is 
denoted by w. 

2. Choice is a function that assigns to each a. G Ags and each \3-class w (with 
w G W) a partition Choice™ ofw given by an equivalence relation which we will 
denote by R'f. Choice must satisfy the following constraint: 

- (IA)k For w G IT, we have that each function s : Ags —)■ V{w) that maps a to 

a member 0 /Choice™ is such that ® of all functions s 

that map a to a member 0 /Choice™ is denoted by Select™). 

For w G IT and v € w, the class of v in the partition Choice™ is denoted by 
Choice™(u). For ru G IT and /? G Ags, we define StateJj = {S' C w; S = 
(^c&Ags-{0} s(«). ^^^ere s G Select™}. 

3. For each a G Ags, is an (epistemic) equivalence relation on W that satisfies 
the following constraints: 

- (OAC)k For w G IT such that v G w, if v u, then v' u for every 
v' G Choice™ (v). 

- (Unif — H)k Let wi,W 2 G IT such that there exist v G Wi and u G W 2 with 

V u. Then for every v' G uR, there exists u' G uR such that v' «q, u'. 

For each u G IT such that w ~q, v, we define the epistemic cluster set of L at v by 
|L](( := (u G v; 3o G L such that o u}. 

4. Valueo and Valuer are (utilitarian) deontic functions that assign to each w G IT 
a value in IR. We use them to define objective and subjective orderings just as for 
BT-models. 

Kripke-estit frames allow us to evaluate the formulas of £ko in ™ analogous fash¬ 
ion to the semantics provided for BT-frames. Therefore, we use the name ‘Kripke-estit 
models’ for the structures that result from adding a valuation function to Kripke-estit 



frames, where the semantics for the formulas of £ko is defined so as to mirror Defini¬ 
tion 3. Kripke-estit models can be used for constructing epistemic utilitarian bi-valued 
BT-models such that both satisfy the same formulas of £ko- 

Definition 6 (Associated BT-frame). 

Let T = {W, Choice, {sSajagAgs) Valuer, Valuer) be a Kripke-estit frame. 
We define the tuple (Tvi/, □, Choice, {~Q,}Q,g^gs, Valuer, Valuer) will be called 
the epistemic utilitarian bi-valued BT-frame associated to T such that 

1. T]v := W U {w; w € TW}U{1W} and \Z is a relation on Ty\/ such that \Z is defined 
as the transitive closure of the union {(w, v);w G W and v € w} U {{W, w);w € 
W}. Notice that \Z is a strict partial order on T\y that satisfies “no backward 
branching” straightforwardly. Since the tuple {Tyy, c) is thus a tree, we can call the 
maximal \Z-chains in Tyy histories. We observe that the definition of \Z yields that 
there is a bijective correspondence between the set of worlds in W and the histories 
ofTyy. If for each v € W, we set hy to be the history {W, v, u}, then we have that 
for any world o G W, o G hy iff o = v. Therefore, we will conveniently identify 
these histories using the worlds at their terminal nodes. In this line of thought, we 
have that for each w € W, the set of histories running through w is equal to 
the set {hy;v G uJ} -since w G hy iff v € w. Similarly, we have that the set H of 
all histories is the same as {hy]v £ W}. 

2. Choice is an agentive choice function for the tree of the above item. In order 

to define such a function, we use the partitions Choice^ (where w £ W and 
a € Ags) in the following manner: for each B € V{W), we will write to 
denote the set {hy] v € B{; this way, we define a function Choice on Ags x 
with extension given by Choice(Q;, v) = for each a G Ags and v G W, 

Choice(Q;, w) = {Cj; Ca G Choice™} for each w & W and each a G Ags, and 
Choice(Q;, IW) = {H{ for each a G Ags. To keep notation consistent, we will 
denote sets of the form Choice(a,u;) by Choice™, so that the choice-class of a 
given hy in Choice™ is denoted by Choice™(h„), as expected.^^ It is important 
to mention that this definition yields that for every S G State™, G State™, 
and that for each U G State™, there exists V G State™ such that U = 

3. For each a G Ags, we define an epistemic relation on Tyy using the set 
}aeAgs- For a given agent a G Ags, we define ~q= {{{w, hy), (w', hy>))] w, w' G 
W and V u'}U{((z, /i^), {z,hf))]z G W}\J{({W,hy), {W,hy>))]v,v' G W}. 
We observe, then, that this definition entails that ~q, is an equivalence relation for 
every a G Ags and that, for a given w & W and L G Choice™, v G [i]™ iff 

K e [L^]™. 

4. Valueo and Valuer are two utilitarian deontic functions, defined such that for 
each hy G H, Valuec)(/i„) = Valuec)(u) and'VBhxesfiy) = Value 5 (u). En¬ 
dowed with these deontic functions, we can define objective and subjective domi¬ 
nance orderings for this structure accordingly. 

Propositions. Let F = (W, i?n. Choice, {«Q,}Qg^gs, Valueo, Valuer) foe a/Tripke- 
estit frame. The tuple (T^y, C, Choice, {~Q,}Qg^gs, Valuer, Valuer) is an epis¬ 
temic utilitarian bi-valued BT-frame. 

** Notice that this implies that, for two worlds v, v' in w, vRfv' iff hy G Choice™ (hy/). 



Proof. It amounts to showing that the tuple, as defined, validates the semantic condi¬ 
tions (NC), (lA), (OAC), and (Unif - H). 

- (NC) is vacuously validated in moment W. It is validated in moments of the form 
W (w G W), since two different histories never intersect in a moment later than w). 
Finally, it is also validated in moments of the form v such that v GW (since there 
are no moments above v). 

- For (lA), we reason by cases: 

a) At moment W, (lA) is validated straightforwardly, since Choice(a,fF) = 
{H} for each a G Ags. 

b) For a fixed moment of the form w (with w G W), let s be a function that 
assigns to each agent a a member of Choice™ = {(Cq,)^; G Choice™}. 
Let Sk '■ Ags —> UaeAgs Choice™ be a function such that Sfe(a) = Cq iff 
s(a) = (Cq,)^. Since Ai validates the constraint (lA)^^:, then we have that 
riaeAgs ^kia) f 0. Take v G flaeAgs Sfc(«)- Then v G Ca for every a G 
Ags. This implies that G (Cq,)^ for every a G Ags, so that PlaeAgs s{a) 

0 . 

c) At moments of the form v such that v G W, if s is a function that assigns to 
each agent a a member of Choice(w, a), we have that s must be constant and 

naeAg.s(«) = {^v}- 

- For (OAC), again we reason by cases: 

a) Assume that Ifu, hf) ~q. {w', hyi) (for w, w' € W). This means that v v'. 
We want to show that for every hu G Choice™ such that € Choice™ (/i^,), 
{w,hu) {w',hyi). Therefore, let /i„ G Choice™(fi„). By definition, we 
have that this means that u G Choice™(r!). Since A4 validates the constraint 
(OAC)k, this last fact implies, with v v', that u v', which in turn yields 
that fw, hu) {w', hy). 

b) For situations with moments of the form v such that v G W, we have that 
(OAC) is met straightforwardly, since for hy, the choice-cell in Choice(a,ti) 
to which hy belongs is just {hy}. 

c) For situations with moment W, (OAC) is also met straightforwardly, since for 
every a G Ags, is defined such that {W, hy) {W, hyf for every pair of 
histories hy , hy/ in H. 

- For (Unif — H), again we reason by cases: 

a) Assume that {w, hy) ~q, {w' , hy') (for w, w' G W). This means that v G w, 

v' G w', and v v' . Let hz G (which means that 2 ; G w). We want 
to show that there exists h G such that Ifu, hz) ~a {w', h). Condition 
{Unif — H)k gives us that there exists z' G w' such that z z', which by 

definiton of means that Ifu, hz) {w', hz'). 

b) For situations with moments of the form v such that v G W, we have that 
(v, hy) is -related only to itself, so that (Unif — H) is met straightforwardly. 

c) For situations with moment W, (Unif — H) is also met straightforwardly, since 
{W, hy) (IF, hy') for every v, v' G IF. 


Let Af be a Kripke-estit model with valuation function V. The frame upon which M. 
is based has an associated BT-frame. If to the tuple of this BT-frame we add a valuation 



function such that V'^{p) = {{w, hw)',w € V(p)}, we call the resulting model the 
epistemic utilitarian bi-valued BT-model associated to Ai. 

Proposition 4. Let M. be a Kripke-estit model, and let A4* denote its associated BT- 
model. For a formula ip of w G W, we have that Ai, w\\-p iff Ai"^, {w, hyf) N 

T- 

Proof Claim 1. For a given Kripke-estit model Ad and a fixed w G W^, we have that 
for every L,N G Choice™, 

- LfNiffL^fN'^andL^N iff ^ N'^ . 

- LfsN iff and L iV iff ■ 

- L G Optimal™ iff G Optimal™, andi € S — Optimal™ iffi^ € S — Optimal 

We will prove Claim l.only for the strict orderings, since these proofs include the ar¬ 
guments needed to show that the statements for ^ and fg also hold. 

- (^) We assume that L ^ N. Let U G State™. We know that U = for 
some V G State™. Therefore, we have that L OV < N nV. But notice that, 
by definition, we have that for each v G w, Valuec)(/i^) = Valuec)(r!), so the 
fact that L DV < N nV implies that HU < N"’" n U. Now, our assumption 
also yields that there should exist Sq G State™ such that N H Sq ^ L D Sq. This 
straightforwardly implies that iV^ n Sq ^ n Sj, so that indeed we have that 

^ N^. 

{g=) We assume that ^ iV^. Let S G State™. We know that n S'^ < 

N'^ n 5*^, so that L n 5” < TV n S'. Similarly, here our assumption yields that 
there should exist Uq G State™ such that N'^ H C/q ^ H Uq- But we have that 
Uq = Vq for some Vq G State™, so that the definition of Valueo implies that 
TV n Vb ^ ^7 n Vb- With this, we have shown that L ^ TV. 

- (=^>) We assume that L ^g TV. Let w' be such that w w', and let (7 G State™ . 

We know that U = for some V G State™'. Therefore, we have that |L]™' n 
V <s [TV]™' n V. We have that for each w G W, Value 5 (Tiu,) = Value 5 (w), 
and that for each pair Vi,V 2 such that vi G [L]™' H V and V 2 G |TV|™' n V, then 
hy^ G [L"^]™' n V'^ and hy^ G [TV^]™' n so that the fact that |L|™' n C <« 
[TV]™' n V then implies that [L^]™' H U <s [TV]™' n U. Now, our assumption 
also yields that there should exist re* such that w «« tc* and Sq G State™* such 
that |TV|™* n Sq ^g |L|™* n Sq. Analogously, this implies that [TV^]™* n S'o ^g 
[L^]™* n Sq, so that indeed we have that L'^ TV^. 

(■<=) We assume that N'^. Let w' be such that w «q, w', and let S G 

State™'. We know that [L^j™ <, [TV^]™'n^^, so that an argument similar 

to the one displayed for the above item renders that [L]™' H S' <« [TV]™' n S. On 
the other hand, there should exist w* such that w «q, w* and Uq G State™* such 
that TV^ n Uq -^s n Uq. But we have that Uq = Vq for some Vb G State™*, so 
that, in an analogous fashion to what we did above, we get that TV n Vb LFVq. 
With this, we have shown that L TV. 

- Straightforward from the above items. 



With that we conclude the proof of Claim 1. For the proof of the main statement, 
we proceed by induction on ip. For the base case, we have that, for a propositional 
letter p and an arbitrary w G W, A4,w IF p iff rc e V(p) iff {w,hiu) G V^(p) iff 
{W, hyj) F p. The cases with the boolean connectives are standard, so let us deal 
with the modal operators. Let w GW and a G Ags. 


- (“□”) 

We have that IF Dp iff for every v G W, A4w,v IF p, which by induc¬ 

tion hypothesis happens iff , {v, hy) F p for every v G W, which happens iff 
Ai"^, {w, hyj) F Dp, since we have that hy G Hyj iff v Gw. 

- (“H”) 

We have that Ai,w IF [a](p iff for every v G W such that wR^v, Ai,v IF p, 
which by induction hypothesis happens iff A4'^,{w,hy) F p for every hy G 
Choice^(hyj), which in turn happens iff (w, hyj) h \a]p. 

- 

We have that A^, w IF KaP iff for every v G W such that w v, A4, v IF p, 
which by induction hypothesis occurs iff A4"^, (v, hy) F p for every hy G H such 
that {w, hy,) (F, hy), which happens iff A(^, {w, hy,) F Kap. 

- (“©[a]”) 

(^) We assume that Ai,w IF ©[a](p. Then for every L G Choice™ such that 
A4,vl'^ P for some vl G L, there exists L' G Choice™ such that L ^ L' and if L” 
L' or L' ^ L", thenAf,w IF p for every r; G L". Let N G Choice™ such 
that A4'^, {w, hz) F p for some hz G N. By induction hypothesis, we get that 
A4,z IF p. The cell N is for some Lg € Choice™. Since z G Lq and 
A4,z IF p, our assumption entails that there exists Li G Choice™ such that 
Lq -< Li and if L” = Li or Li ^ L” then A4,v IF p for every v G L”. We 
claim that Lj is the choice cell at moment w that will witness to the fact that 
Ai"^, {w, hy,) F ©[ajp. In order to show this, we first notice that the first item of 
Claim 1 renders that Lq -< Lj. Now, we also have that for every hy G Lf, v G Li 
(observe also that v G w and therefore A4,v IF (p; in this way, we have that for 
every hy G Lj, the induction hypothesis gives that AI^, {W, hy) \= p. Finally, let 
G Choice™ such that Lj ^ By the first item of Claim 1, this last con¬ 
dition implies that Li ^ V. Take hy G V^. We know that v G V, so that the fact 
that Li ^ V implies with our assumption that Ai,v Ih p; therefore, the induction 
hypothesis gives that A4^, {W, hy) F p. With this, we have shown that our claim is 
true and that Ai^, {w, hy,) F ©[ajp. 

(<J=) We assume that Ai^, {w, hy,) \= ©[a](p. Then for every G Choice™ such 
that Ai^, {W, hz) F p for some hz G LA, there exists Lf’ € Choice™ such that 
LA and if U = Lf or ^ U, then A4^, (w, hy) F p for every hy G U. 
Let N G Choice™ such that A4,v p for some v G N. By induction hypothesis, 
we get that A4^, (w, hy) F p. We have that N'^ G Choice™. Therefore, our as¬ 
sumption entails that there exists Lj G Choice™ such that iV^ ^ Lf and if U = 
Lf or ^ U, then AI^, {w, hy) F p for every hy G U. Here, we claim that Li 
is the choice cell that will witness to the fact that Ai,w IF ©[ojp. In order to show 
this, we first notice that the first item of Claim 1 renders that N < Li. Next, for 
every v G Li, hy G Lf and therefore AI^, {w,hy) F p-, in this way, we have 



that for every v € Li, the induction hypothesis gives that Ih ip. Finally, let 
L € Choice™ such that Li ^ L. By the first item of Claim 1, this last condition 
implies that ^ Take v & L. We know that hy G , so that the fact that 
Lf ^ implies with our assumption that (w, hy) 1= ip', therefore, the in¬ 
duction hypothesis gives that Ji/[,v IF ip. With this, we have shown that our claim 
is true and that My,, w IF ©[a](p. 

- (“05 [a]”) 

(^) We assume that M,w IF © 5 [q;]i^. Then for every L G Choice™ such that 
A4,vl ip for some vl G [T]™' (with w «q, w'), there exists L' G Choice™ such 
that L -<s L' and if L” = L' or L' L”, then for every w” such that w «q, w”, 

M,v IF for every v G Let N G Choice™ such that {w', h^) ^ 

ip for some hz G [-/V]™ (with w ~q, w'). By induction hypothesis, we get that 
A4,z IF ip. The cell N is for some Lq € Choice™. Moreover, our defini¬ 
tion of in terms of entails that z G |Lo]q ^ so that the fact that A4,z 
ip entails with our assumption that there exists Li G Choice™ such that Lo ©s 
Li and if L" = Li or Li ©g L" then for every w” such that w sSq, w”, A4, v IF 
ip for every v G |L"]™”. We claim that is the choice cell at moment w that 
will witness to the fact that {w, hy,) F © 5 [q;](/?. In order to show this, we 
first notice that the second item of Claim 1 renders that Lj ©g Ly[. We also have 
that for every w" such that w ~q. w" and hy G [L^]™”, v G |Li]q” and there¬ 
fore Ai,v IF ip; in this way, we have that for every w" such that w w" and 
every hy G [Li']™ , the induction hypothesis gives that (w”,hy) \= ip. Fi¬ 
nally, let G Choice™ such that Lf ©g V'^. By the second item of Claim 
1 , this last condition implies that Li ©g V. Take w" such that w ~q, w", and 
take hy G . We know that v G |y]a”, so that the fact that Li ©g V im¬ 

plies with our assumption that A4,v IF ip; therefore, the induction hypothesis gives 
that A^^, (w", hy) \= if. With this, we have shown that our claim is true and that 
M'^,{w,hy,) ^ Qs[a]ip. _ 

(<J=) We assume that , {w,hy,) \= Then for every G Choice™ 

such that Ai^, {w, hz) F ip for some hz G (with w w'), there exists 

Lf G Choice™ such that ©g and if U = Lf or L'^ ©g U, then for 
every w" such that w w", A4^, (w",/i„) F ip for every hy G [C]™”. Let 
N G Choice™ such that Ai,v IF for some w G |W]™" (with w w')- By 
induction hypothesis, we get that A4^, {w', hy) F ip. The fact that w w' implies 
that w wh Since hy G [A^^]™\ then, we know that there exists Lf G Choice™ 
such that ©g Lf and if U = Lf or Lf :<s U ,then for every w" such that 
w ~Q, w",Ai^, {w",hu) F ip for every hy G [C]™”. Here, we claim that Lx 
is the choice cell that will witness to the fact that Ai,w IF © 5 [q;](^. In order to 
show this, we first notice that the second item of Claim 1 renders that N ©g Li. 
Next, for every w” such that w «q, w” (which implies that W tc") and every 
V G |Li]™”, hy G [LfX^" and therefore Ai"^, {w", hy) F ip; in this way, we have 
that for every for every w” such that w sso, w” and every v € |Li]™”, the induction 
hypothesis gives that A^, IF i^. Finally, let L G Choice™ such that Li ©g L. By 
the second item of Claim 1 , this last condition implies that Lf ©g Take w' 



such that w sSq, w' (which implies that w w'), and take v G |^]a- We know 
that hv G so that the fact that Lj implies with our assumption that 

(w", hy) \= (fi', therefore, the induction hypothesis gives that A^, v Ih (f. With 
this, we have shown that our claim is true and that A^, ru Ih © 5 [q!]v 3 - 

B.l Canonical Kripke-estit models 

We will show that the proof system A is complete with respect to the class of Kripke- 
estit models (and thus use Proposition 4 to show completeness with respect to the class 
of epistemic utilitarian bi-valued BT-models). The strategy is to build a canonical struc¬ 
ture from the syntax. 

Definition 7 (Canonical Structure). The tuple 

M = Choice, {WajaeAgsiValueo, Valuer, "P) 

is called a canonical structure for A iff 

- = {u;;u;/sa A-MCS}. 

- is a relation over defined by the following rule: for w,v G W^, wBuv 
iff for every tp, Dtp G w ^ <p G v. For w G W^, the set {u G W^;wRav} is 
denoted by w. 

- Choice is a function that assigns to each a and w a subset ofVifw), which will 
be denoted by Choice^ and defined as follows: let Rff be a relation on w such 
that for w,v G W^, wRffv iff for every p, [a\p G w ^ p G v; if we take 
Choice™(u) = {u Gw; vR^u}, then we set Choice™ = Udg® Choice™ (v). 

- For each a G Ags, «q- is an epistemic relation on given by the following rule: 
for w,v G W^, w «Q, V ijffor every p, K^p G w ^ p G v. 

- Valueo and Valuer are defined as follows. For a G Ags and w G W^, we first 
define two sets: 

= {VAT',®[a]p G w} 
r™ = {Kap;Qs[a]p G w}. 

If we take 17™ = IJae^igs A™ = lJae. 4 gs define the deontic 

functions by 


Valuec)(ri;) 


1 #r™ C w 
0 otherwise. 


Valuer (w) 


1 if r™ C w 
0 otherwise. 


- V is the canonical valuation, defined such that w G V{p) iffp G w. 


Proposition 5. The canonical structure Ai for A is a Kripke-estit model. 

Proof. We need to show that the tuple {W^, i?n, Choice, {f^a}aGAgs, Valueo, Valuer) 
is a Kripke-estit frame, which amounts to showing that the tuple validates the five items 
in the definition of Krioke-estir models. 



1. It is clear that R\j is an equivalence relation, since A includes the S5 axioms for □. 

2. Since A includes the S5 axioms for [a] (for each a G Ags), we have that i?™ 
is an equivalence relation for each a G Ags and w G W^. Moreover, since A 
includes Dp —> \a]p as an axiom schema, we have that Q Ra- In this way. 
Choice indeed assigns to each a and W a partition of w. Now, we must verify that 
A4 validates the constraint (lA);^. In order to do that, we need two intermediate 
results: 

a) For a fixed lu* G W^, we have that w G uR iff Otp G w*} C w. 

(=>) Let w G uu (which means that Wt,Rijw). Take (p a formula of £ko such 
that 'Onp G tu*. Since ru* is closed under Modus Ponens, the (4) axiom for □ 
implies that G w* as well. Therefore, by the definition of we get that 

€ w. 

(<J=) We assume that {□f/'; G w*} C w. Take ip a formula of £ko such that 
G w*. By our assumption, we get that G W. Since w is closed under 
Modus Ponens, the (T) axiom for □ implies that p G w as well. In this way, we 
have that the fact that □(/? € w* implies that p G w, which means that w^,Raw 
and w G up. 

b) For a fixed w* € and s : Ags -G V{up) a function that maps a to a member 
of Choice™* such that Vs(^a) € s{a), we have that w G s(a) iff 2 \s(q) = 

[a]ip G ^^(a)} Cw. _ 

(^) Let w G s(a) (which means that Us(Q)i?™*w). Take p a formula of £ko 
such that [a\p G Vs{a). Since is closed underM oc/ms Ponens, the (4) axiom 
for [a] implies that [«][«]</? G Vs{a) as well. Therefore, by definition of i?™*, 
we get that \a\p G w. 

(<t=) We assume that Z\s(q,) = {[q;]^/>; [a]ip G fs(Q)} C w. Take p a formula of 
£ko such that [a]p G By our assumption, we get that \a\p G w. Since 

w is closed under Modus Ponens, the (T) axiom for \a] implies that p G w as 
well. In this way, we have that the fact that \a\p G Vs(a) implies that p G w, 
which means that Vs(a)Ra*w and w G s(a). 

Next, we will show that for a fixed lu* G and s : Ags —>■ P(up) a function 
that maps a to a member of Choice™* such as in item b) above, we have that 
Uaehgs U {□V’; □t/’ G w*} is Al-consistent. 

First, we will show that UaGhgs ^s(a) is consistent. Suppose that this is not the 
case. Then there exists a set {pi ,..., (/?„} of formulas of £ko such that G 

Us(q.) for every 1 < i < n and 

P {[ai]piA---/\[an]pn)^ ( 1 ) 

Without loss of generality, we assume that at ^ aj for all j ^ i such that j, i G 
{1,... ,n} -this assumption hinges on the fact that any stit operator distributes over 
conjunction. Notice that the fact that [ai]pi G fs(ai) for every 1 < i < n implies 
that Oicti] G w* for every 1 < i < n. Since ru* is closed under conjunction, we 
also have that <}[q!i](/?i A • • • A G w*. 

By the independence-of-agents axiom, we have that 


h ()[ai]pi A • • • A <)[an]pn <){[ai]px A • • • A [an]pn). 


( 2 ) 



Therefore, equations (2) and (1), imply that 


h 0[ai]</3i A • • • A 0 [q;„](/9„0_L. (3) 

But this is a contradiction, since we had seen that A • • • A 0 [q;„](/ 9 „ G tu*, 

and Wi, is a /1-MCS. Therefore, UaeAgs ^s(a) is consistent. 

Next, we show that the union IJaeAgs ^s(a) U {DV'; € w»} is also consistent. 

Suppose that this is not the case. Since IJaeAgs ^s(a) is consistent, there must exist 
sets {(pi ,..., <pn} and { 0 i,..., 9^} of formulas of £ko such that [ai]ipi € 
for every 1 < i < n, D9i G w* for every 1 < i < to, and 


h {[ai]^! A • • • A [an]<Pn) A (D^i A • • • A UOm) -L- (4) 

Let 6 = 6*1 A • • • A 0m. Since □ distributes over conjunction, we have that 00 = 
□01 A • • • A 00^,, where it is important to mention that since w* is a yl-MCS, then 
□0 G . In these terms, (4) implies that 

h A ••• A [«„](/?„)-> ^00. (5) 

Once again, we assume without loss of generality that ^ Uj for all j ^ i 
such that j,i G {1,... ,n}. Analogous to the procedure we used to show that 
UaeAgs 4is(a) is Consistent, (5) implies that 

h (>[ai]v3i A • • • A 0[a„](/?„O^D0. (6) 

This entails that <>^n0 G tu*, but this is a contradiction, since the fact that 00 G w* 
implies with the (4) axiom for □ that 000 G w». 

Now, let M* be the yl-MCS that includes UaeAgs ‘^s{a) U {□V’; € w*}. By the 

intermediate result a), it is clear that u* G uT. By the intermediate result b), it is 
clear that m* G s(a) for every a G Ags. Therefore, we have shown that for a fixed 
w* G W, we have that each function s : Ags —)■ V{wt) that maps a to a member 
of Choice™* is such that HaGAgs which means that Ai validates the 

constraint 

3. Since the axiom system yl includes the S5 axioms for for each a G Ags, we 
have that is an equivalence relation for each a G Ags. We must now verify that 
A4 validates the constraints (OAC)iG and (Unif — H)ig. 

i) For (OAC)i^, fix w* G and a G Ags. We assume that v,u G wZ are such 

that V «Q, u. Let v' G Choice™* (v). This means that vRav'. We want to show 
that v' «Q, u, so let be a formula of £ko such that Kap G u'. By the (4) 
axiom for K^, we have that G v'. Similarly, since all substitutions 

of the (OAC) axiom lie within v' and it is closed under Modus Ponens, we 
get that \a]KaP also lies in v'. Since v'RaV, this implies that Kap G v. 
Therefore, our assumption that v «q, u entails that (p G u. With this, we have 
shown that the fact that K^p G v' implies that p G u, which means that 
v' u- 

ii) For (Unif — E)k, fix W\,W 2 G W^. We assume that there exist v G W[ and 
u G W 2 such that v «q, u. Take v' G uR. 



We will show that u” = {ip] Kaip € v'} U {Oip; Dip G u} is consistent. In 
order to do so, we will first show that {ip] K^ip € v'} is consistent. Suppose 
for a contradiction that it is not consistent. Then there exists a set {ipi,... ,ipn} 
of formulas of £ko such that {ipi ,..., ipn} C {ip; K^ip G v'{ and h i/)i A - • • A 
_L (a). Now, the fact that {ip \,..., ipn] C {ip] Kaip G v'{ means that 
Kaipi G v' for every 1 < f < n; by necessitation of Ka and its distributivity 
over conjunction, we get that (a) implies that h Kaipi A • • • A Kaipn —> Ka-i-, 
but this is a contradiction, since v' is a yl-MCS which, being, includes Kaipi A 
• • • A Kaipn- 

Next, we show that u” = {ip] Kaip G v'} U {Dip] Dip G u} is also consistent. 
Suppose for a contradiction that it is not consistent. Since {ip] Kaip G v'} is 
consistent, there must exist sets {(/?!,...,(/?„} and {6i,... ,0m} of formulas of 
£ko such that {ipi,..., ipn} C {ip; Kaip G v'}, D9i G W 2 for every 1 < f < 
m, and h {(pi A • • • A (/?„) A iD6i A • • • A D6m) —> -L (a). Let 9 = 9iA - ■ ■ A6m 
and (f = (fii A ■■■ A Pn- Since □ distributes over conjunction, we have that 
D9 = D9i A ■ ■ ■ A D9m, where it is important to mention that since t 6 is a 
yl-MCS, then D9 G u and G m as well (*). In this way, (a) implies that 
!-(/?—)■ -AD9 and thus that h ()p -A 0^00 (b). Now, by necessitation of Ka, 
we get that (b) implies that h Ka()p -A Ka(}^D9 (c). Notice that the facts 
that KaPi G v' for every 1 < i < n, that Ka distributes over conjunction, 
and that v' is a yl-MCS imply that KaP = KaPi A ••• A KaPn G v'. The 
fact that v' G wT = v implies that (}KaP G v, so that (c) and the facts that 
V contains all substitutions of the {Unif — H) axiom and is closed under 
Modus Ponens entail that KaOp G v. Now, this last inclusion implies, with 
our assumption that v u, that ()p G u, which by (b) in turn yields that 
<>^□0 G u, contradicting ( 7 k-). Therefore, u” is consistent. 

Finally, let u' be the yl-MCS that includes u”. It is clear from its construction 
that u' Gu = W 2 and that v' u', 

With this, we have shown that Ai validates the constraint (Unif — . 

4. Both Valueg) and Valuer are well defined functions with range within R. 

As is usual with canonical structures, our objective is to prove the so-called truth 
lemma, which says that for every formula p of £ko und every w G W^, we have 
that Ai’^,w IF p iff p G w. This is done by induction on p, and the inductive step for 
each modal operator requires previous results (such as the important existence lemmas). 
In the case of D,Ka, and [a], these previous results are standard (Lemma 1 below). 
However, in the case of ©[a] and ©5 [a], the previous results require a lot of work. In 
Lemmas 2 and 3 below, we pinpoint the main claims that we need to show the inductive 
step of the truth lemma for the ought operators. Each item in these two lemmas is shown 
by a series of sub-lemmas, and the reader is advised to carefully go over the full proofs 
provided in the supplement to the present work. 

Lemma 1 (Existence). Let Ai be the canonical Kripke-estit model for A. Let w G W^. 
For a given formula p of C^o, the following hold: 

1. Dp G w ijf p G V for every v Gw. 

2. [oi]p G w iff p G V for every v Gw such that wRaV. 



3. Kaip G w ijfip G V for every v G such that w v. 

Proof. Let w G W^, and take ip a formula of £ko- 

1. (=>) We assume that 'Onp G w. If v lies within w, this means that wRuv holds. 
Therefore, (p G v. 

(<;=) We work by contraposition. Assume that 'Onp ^ w. We will show that there is a 
world in w such that ip does not lie within it. For this, let v' = {f] □t/’ G w}, which 
is consistent by virtue of the following argument: suppose for a contradiction that v' 
is not consistent; then there exists a set ..., t/)„} of formulas of £ko such that 
{tpi, ■ ■ ■ ,fn} v' and h ■01 A - • •A'0n —>■ -L (a); now, the fact that {-fi ,..., 0„} C 
v' means that 'Oifi G w for every 1 < i < n; due to necessitation of □ and its 
distributivity over conjunction, we get that (a) implies that h □■0i A • • • A □■0„ —> 
□_L, but this is a contradiction, since ru is a yl-MCS which, being closed under 
conjunction, includes 'Oi'fi A • • • A □■0„. Now, we define v” = u' U {^ip}, and 
show that it is also consistent as follows: suppose for a contradiction that it is not 
consistent; since v' is consistent, we have that there exists a set { 0 >i,..., 0 >„} of 
formulas of £ko such that ..., 0>„} C v' and h fi A ■■■ A ipn A —•(p —>■ _L, 
which then implies that h A • • • A —)• v? (b); due to necessitation of □ and its 
distributivity over conjunction, we get that (b) implies that h □0>i A • • • A Ofn —> 
'Oap (c); but notice that since ru is a yl-MCS, then □■0i A • • • A □V’n G w, so that (c) 
and the fact that w is closed under Modus Ponens entail that 'Oap G w, contradicting 
the initial assumption that 'Oap ^ w. Finally, let v be the yl-MCS that includes v”. 
On one hand, it is clear from its construction that p fiv and that wRav. Thus, we 
have shown that assuming that 'Oip ^ w implies the existence of a u G u; such that 

p ^ V. 

2. (=a) We assume that [a]p G w. Let v Gw such that wRaV. The definition of Ra 
straightforwardly gives that p G v. 

(■G=) We work by contraposition. Assume that [c(\p w. We will show that there 
is a world v in uJ such that wRaV and such that p does not lie within it. For 
this, let v' = {0; [a]0 G w}, which is consistent by a similar argument than 
the one introduced in the proof of the above item: suppose for a contradiction 
that v' is not consistent; then there exists a set {0 >i,..., 0>„} of formulas of £ko 
such that {'01 ,... ,0„} C v' and h A • • • A 0„ —> -L (a); now, the fact 
that 101 ,...,0„} C v' means that [a]0i G w for every 1 < t < n; neces¬ 
sitation of [a] and its distributivity over conjunction yield that (a) implies that 
h [q;]0i a • • • a [q;] 0„ —)■ [a]-L, but this is a contradiction, since w is a yl-MCS 
which, being closed under conjunction, includes [q;]0i A • • • A [q;]0„. Now, we de¬ 
fine v” = v' U {~'p}, and we show that it is also consistent as follows: suppose 
for a contradiction that it is not consistent; since v' is consistent, we have that there 
exists a set {0i,... ,0„} of formulas of £ko such that {0i,... ,0„} C v' and 
h 01 A • • • A 0„ A —•p -G _L, which then implies that h 0i A • • • A 0„ —> (b); 

due to necessitation of [a] and its distributivity over conjunction, we get that (b) 
implies that h [q;]0i A • • • A [q;]0„ —>■ [c(\p (b); but notice that since w is a A-MCS, 
then [a]0i A • • • A [q;]0„ G w, so that (b) and the fact that w is closed under Modus 
Ponens entail that [a]p G w, contradicting the initial assumption that \a\p ^ w. 



Finally, let v be the yl-MCS that includes v". On one hand, it is clear from its con¬ 
struction that Lp ^ V and that wRaV, by definition of R^. On the other, axiom {A2) 
renders that actually r; € wJ (if 06* € w, then [a]0 G w and thus 0 G v). Therefore, 
we have shown that assuming that [c(\{p ^ w implies the existence of a G wJ such 
that wRaV and (p ^ v. 

3. (=4>) We assume that Kap> G w. Let v G such that w v. The definition of 
«Q, straightforwardly gives that ip G v. 

(<=) We work by contraposition. Assume that Ka‘P ^ w. We will show that there 
is a world v in such that w v and such that (p does not lie within it. 
For this, let v' = {'tp; Katp G w}, which is consistent by an argument similar to 
the one introduced in the proofs of the above items: suppose for a contradiction 
that v' is not consistent; then there exists a set {'tpi ,..., of formulas of £ko 
such that {ipi,... ,ipn} C v' and h t/>i A • • • A f/'n —> -L (a); now, the fact that 
{^ 1 ,..., ipn} Q v' means that Katpi G w for every 1 < i < n; necessitation of 
Ka and its distributivity over conjunction yield that (a) implies that F A • • • A 

Ka'tpn -G Ka-i-, but this is a contradiction, since w is a A-MCS which, being closed 
under conjunction, includes A • • • A Kaipn- Now, we define V” = v'\J {-'If} 

v" = v' U {^p>}, and we show that it is also consistent as follows: suppose for 
a contradiction that it is not consistent; since v' is consistent, we have that there 
exists a set of formulas of £ko such that {ipi,... ,'ipn} Q v' and 

h 'tpi A ■■■ A 'ipn -A -L, which then implies that h 'ipi A ■■■ A tpn -A p) 

(b); due to necessitation of and its distributivity over conjunction, we get that 
(b) implies that h Kaipi A ■ • • A Kaipn -A Kap (b); but notice that since w is a 
yl-MCS, then Kaipi A • • • A Katpn € w, so that (b) and the fact that w is closed 
under Mot/ws Ponens entail that Kap G w, contradicting the initial assumption that 
Kap ^ w. Finally, let v be the A-MCS that includes v”. On one hand, it is clear 
from its construction that p ^ v and that w v, by definition of Thus, we 
have shown that assuming that K^p 4- implies the existence of a r; G such 
that w «Q, V and p ^ v. 

Lemma 2 (Existence for objective ought). Fix a G Ags and w G W^. 

(a) For every formula p of Lko> ©[q;]i^ & w iff [a\p G v for every v Gw C such 
that C V. 

(b) For every v G Choice“(w), C w iff Sf C v 

(c) S'f C w ij^Choice^jw) G Optimal^. 

(d) For each L G Choice™ — Optimal™, there exists L' G Optimal™ such that L -< 
L'. Thus, A4, w IF ©[a]<y9 iff VL G Optimal™, A4, v IF pfor every v G L. 

Proof (a) Claim 1. Let A4 be the canonical Kripke-estit model for A. We have that 
the following hold: 

- For w* G FF^, w G wF iff {□'*/'; nV" € w*} C w. 

- For any pair of agents a, a G Ags and any formula p of £ko> ©[q:]v? —>■“'© 

G w for every w G FF'^. We will refer to this property as “consistency 
of ought-to-do”. 

The proof of the first item goes as follows: (=>) Let w G wG (which means that 
w^Ruw). Take p a formula of £ko such that Uip G w*. Since w* is closed under 



Modus Ponens, the (4) axiom for □ implies that nDtp € w* as well. Therefore, by 
definition of we get that 'Onp G w. (<=) We assume that {□'0; □V' G it'*} C w. 
Take ip a formula of £ko such that 'Onp G w*. By our assumption, we get that 
□(/? G W. Since w is closed under Moc/ms Ponens, the (T) axiom for □ implies that 
ip G w as well. In this way, we have that the fact that □(/? G lu* implies that (p G w, 
which means that w^,Raw and w G uh. 

The proof of the second item is a direct consequence from the fact that, for a, f3 G 
Ags and ip a formula of Ckq, we have that h ©[a](p —>■ -• © [I3\^p. This comes 
from the fact that, for a,/3 G Ags, h ©[a](/? A ©[/3]-'(/? —)■ _L -which can be seen 
by applying axioms {Oic), {IA), and axiom (T) for [a] and [/3]. 

Now we proceed with the proof of the main statement. 

(=p) The definition of implies straightforwardly that [c(\p G v. 

(■G=) We work by contraposition. Suppose that ©[a](p ^ w. 

- We first show that is consistent. Suppose that is not consistent. Then 

there is a set {pi ,..., Pn] of formulas of £ko such that (D[c(\pi G w for every 
1 < i < n and h A • • • A —>■ -L. This last thing implies that 

h ([a](pi A • • • A [a\pn-i) —> -^[a]pn, so that if we take 'll) = pi f\ - ■ ■ pn-i, 
then distributivity of [a] over conjunction gives us that h [«]■)/)—>■ -^\a\pn- By 
necessitation of the ought-to-do operator and its axiom {K), we get that 

h ©[«]([«]i/j)©[a][a](p„). (7) 

First, we notice that since w is a yl-MCS, then ©[a]'0 G w (by distributivity 
of ©[a] over conjunction). The fact that (£)[a]tp G w implies with axiom (Gl4) 
that ©[a]([a]i/)) G w, which with equation (7) implies that ©[«](-■ [a](p„) G w. 
However, the fact that (D\a\pn G w implies that ©[q;]([q;](/?„) G w. Thus, we 
both have that ©[a](-i[a](p„) G w and that ©[q;]([q;](p„) G w, which is a 
contradiction (by consistency of ought-to-do). 

- Next, we observe that the union U {□i/'; 'Eip G tu} is also consistent. Sup¬ 

pose that U {□i/’; □i/' G luj is not consistent. Since E^ is consistent, we 
have that there must exist sets {pi,..., pn} and { 0 i,..., 0^} of formulas of 
£ko such that G w for every 1 < i < n, 06*^ G w for every 1 < i < m, 

and 

h {[a]pi A • • • A [a\pn) A (□('i A • • • A UOm) -G _L. ( 8 ) 

Let 6 = 6iA- ■ ■ A9m- Since □ distributes over conjunction, we have that EiO = 
□ 6*1 A- • -ADOm, where it is important to mention that since w is a H-MCS, then 
□ 6 * G w. In these terms, ( 8 ) implies that h A • • • A [ajtpn) —> ~^Od. By 

necessitation of the ought-to-do operator, its axiom (K), and its distributivity 
over conjunction, we get that 

h (©[a]([a](^i) A • • • A ©[a]([a](p„)) ©[a](^D 6 »). (9) 

We have that ©[Q;]([Q;]v 3 i) G w for every 1 < i < n. Therefore, their con¬ 
junction also lies in w, which by (9) entails that ©[q;](^D 6 I) G w. However, 
we had also observed that 06* G w, which by axiom (4) for □ implies that 
□□ 6 * G w and therefore that ©[q;](D0) G w. We have struck a contradiction, 
since we cannot have that both ©[a] (^06*) and ©[a] ( 00 ) lie in w (consistency 
of ought-to-do). Therefore, 17“ U {□i/’; Otp G w} is consistent. 



- Finally, we observe that the union 17“ U G iw} U is also 

consistent: 

First, recall that we are assuming that ©[q;]i^ ^ w. Now, suppose that 17“ U 
{Oip; G w} U is not consistent. Since 17“ U {O-tp; Dip G w} is 

consistent, then there must exist sets {ipi, ..., ipn} and {6i,..., 0m} of for¬ 
mulas of £ko such that ©[a](/?i G w for every 1 < t < n, DO^ G w for every 
1 < i < m, and 

h ([q;]</3i a • • • a [a]ipn) A (D^i A • • • A D6m) A -^[a]ip —)■ ©. (10) 

Let 9 = 01 A • • • A 9m- Since □ distributes over conjunction, we have that 
□0 G w. Now, (10) implies that 

h ([a](pi A • • • A [q;]v3„) A 00-> (11) 

By necessitation of the ought-to-do operator, its axiom {K), and its distributiv- 
ity over conjunction, we get that 

H (0[a]([Q]<Pi) A • • • A 0[a]([a]c/3„)) A 0[a](n6>) -> 0[a]([a]«y3). (12) 

Similarly as above, we have that ©[a]([Q;](pi) G w for every 1 < i < n. There¬ 
fore, their conjunction also lies in w. At the same time, 00 G w implies that 
□□0 G w (by axiom (4) for □), and therefore that ©[a](00) G w. There¬ 
fore, (12) entails that © [a] ([a] (/?) G w. But axiom (T) for [a], necessitation for 
the ought-to-do operator, and {K) for the ought-to-do operator then imply that 
©[q;]i^ G w, and this contradicts the previously shown fact that ©[q;]v3 ^ w. 
Let u be the A-MCS that includes 17“ U {Dip] Dip G w} U -^[a]ip. It is clear that 
u Gw (by the first item of Claim 1), that 17“ C u, and that [a\(p ^ u. 

(b) (^) Assume that 17“ C w, and let [a](p G 17“. By assumption, we have that 
[c(\‘p G w. Since w is a A-MCS, axiom (4) for [a] implies that [q!]([q!](p) G w as 
well. With the fact that wR]}v, this implies that [a\Lp G v. Therefore 17“ C v for 
every v G ChoiceQ(rt;). 

(<;=) Analogous. 

(c) Claim 2. For w G the set {u G w] Valuec)(t6) = 1} is not empty. 

(Proof of claim) 

Let w G W^. We will first show that UaeAgs consistent. Let w G W^. Sup¬ 
pose that UaeAgs ^ot Consistent. Then there must exist a set {(/9i,..., (/?„} 

of formulas of £ko such that (D[ai](fi G w for every 1 < i < n and 


h A • • • A [a„]v2„)-)> _L. (13) 

Without loss of generality, we assume that ai ^ aj for all j ^ i such that j, i G 
{1,... ,n} -this assumption hinges on the fact that any stit operator distributes 
over conjunction. Since w is a A-MCS, then the fact that (D[ai](fi G w for every 
1 < 2 < n implies that (©[aijtpi A • • • A ©[q;„](/?„) G w . Notice then that axiom 
{Oic) yields that 


h (©[ai](/9i A • • • A ©[q;„](/?„) -)> <)[ax]‘-pi A • • • A 


( 14 ) 



By axiom {IA), we have that 


h 0[ai](pi A • • • A 0[a„](An 0([ai]v5i A • • • A [«„](/?„). (15) 

Therefore, equations (14), (15), and (13) imply that 

h (©[q;i](/?i A ••• A ©[«„](/?„)-> 0_L. (16) 

But this is a contradiction, since we had that (©[ai](/?i A • • • A ©[an]</5n) € w, and 
w is a yl-MCS. Therefore, IJae^gs consistent. 

Next, we show that the union Ti'™ U {□-!/>; □■)/) € w} is also consistent. 

Suppose that this is not the case. Since IJaeAgs consistent, there must exist 

sets {(y9i,..., (p„} and {9i,... ,6jn} of formulas of £ko such that (l)[ai]Lpi G w for 
every 1 < z < n, 00^ G w for every 1 < z < m, and 


h i[ai]<fi A • • • A A (D^i A • • • A Oe^n) -L. (17) 

Let 0 = 6*1 A • • • A 9m. Since □ distributes over conjunction, we have that 00 = 
□ 6*1 A • • • A 06*^.Since zu is a Tl-MCS, then G w. In these terms, (17) implies 
that 

h ([ai](pi A • • • A ^06*. (18) 

Once again, we assume without loss of generality that ^ aj for all j ^ i 
such that j,i G {1,... ,n}. Analogous to the procedure we used to show that 
UaeAgs Consistent, (18) implies that 

I- (©[q;i](Pi a • • • a ©[a„](/3„) O^D0. (19) 

This entails that G w, but this is a contradiction, since 00 G w implies 

with axiom (4) for square that □□(* G w. Let u be the A-MCS that includes 
UaeAgs U {□z/'i ^'i/' G w}. It is dear that u G w and that Valuec)(zz) = 1. 
Therefore, the set {u G w;Va.lu.eo{u) = 1} is not empty. Therefore, we have 
shown that Claim 2. is true. 

We proceed with the proof of the main statement. Recall that Optimal™ is defined 
as {L G Choice™; there is no L' G Choice™ such that L A L'}. 

Let w G W^. (^) Let w G such that Z'™ C w. We will show that for 
every L G Choice™ and S G State™, S' n L A S n Choice™(zu), which would 
straightforwardly imply that Choice™(zu) G Optimal™. For this, it suffices to 
show that if Valueojzz*) = 1 for some zz* G S n L, then Valuec)(o) = 1 for 
every o G S n Choice™(zu). Therefore, take L G Choice™, S G State™, o G 
S n Choice™(zu), and zz* G S n L such that Valuec)(zz*) = 1. This last thing 
means that U/ 36 Ags ^0 Qu.^. 

As an intermediate result, we observe that for every v G S, U/ 3 GAgs-{a} ^0 — 
let V G S', since zz*, z; both lie in S G State™, this means that v G Choice^(zz*) 
for every /3 G Ags — {a}, so that item (b) above implies that S'p C v for every 
l3GAgs- {a}; therefore, U/ 3 GAgs-{a} ^0 Qv. 

We want to show that Valuec)(o) = 1. Since o was taken in S n Choice™(zz;), 
then, on one hand, our intermediate result gives us that U/ 3 eAgs-{a} 



the other, we have that o € Choice“(t(;) implies by item (b) above that C o. 
With these two facts, we conclude that lJ/ 3 Gyigs ^0 — O’ '^hat Valuer (o) = 1. 
(<t=) We work by contraposition. Suppose that E'^ % w. By item (b) above, this 
implies that % o for every o G Choice™(re), so that Valuec)(o) = 0 for 
every o G Choice™(u;). We will show that there exists an action in Choice™ that 
dominates Choice™(li;). 

From Claim 2. we know that the set {rt G w; Valuec)(t6) = 1} is not empty. 
Take tt* G {m G u;Va.lneo{u) = 1}. We will show that Choice™(M*) is the 
action that we are looking for, i.e., we will show (*) that for every state S G 
State™, S n Choice™(ru) < S' n Choice™(u*), and (**) that there exists a 
state S* such that S* n Choice™(M*) S* n Choice™('u;). For (*), take any 
S G State™. We want to show that S n Choice™(r(;) < S n Choice™(M*). 
Since for any o G S n Choice™(w) we have that Valuec)(o) = 0, this en¬ 
tails rather straightforwardly that S n Choice™('u;) < S n Choice™(M*). For 
(**), we set S„, = n/ 36 Ags-{a} ChoiceJ, (u*). It is clear that S„, G State™ 
and that m* G S„, n Choice™(t6*). Similarly, we have that (lA);^ implies that 
Su, n Choice™(r(;) ^ 0. Since for any o G S„„ n Choice™(w) we have that 
Valuec)(o) = 0, it is clear that S„„ n Choice™(u*) ^ S„„ n Choice™(w). Now, 
our results (*) and (**) together render that Choice™ (w) ^ Choice™ (w*), and this 
implies that Choice™(w) ^ Optimal™. 

(d) Let w G and L G Choice™ — Optimal™. By Claim 2., there exists m* G w 
such that Valuec)(M*) = 1. Firstly, from item (c), we get that Choice™ (m*) G 
Optimal™. The fact that L G Choice™ — Optimal™ implies with item (b) that 
Valuec)(w) = 0 for every w G L, so that S' n L < S' n Choice™(t6*) for every 
S G State™ (*). On the other hand, if we take S* to be the unique state in State™ 
such that M* G S*, then it is clear that S* n Choice™(M*) ^ S* n L (**). Our 
results (*) and (**) together render that L -< Choice™ (m*). 

As for the previous results for © 5 , we first observe that for a G Ags, the S5 axioms 
for the combination of operators are theorems of A, which entails that the relation 
i?no is an equivalence relation. For each w G IF^, we will denote by Q;fe[w] the set 
{v G IF"^; KaD(p G w ^ (p € v}. Since the proof of the following lemma is the most 
important new development of the present work, we include its basic steps (the details 
are provided in the supplement). 

Lemma 3 (Existence for subjective ought). Fix a G Ags and w G W^. 

(a) For every formula ip of Lko, G w ijf K^ip G v for every v G ak[w] such 

that C V. 

(b) For every v G |Choice™(r(;)]™' (w w'), F™ C w iff Ff^ C v. 

(c) F™ C w Choice™(w) G S - Optimal^. 

(d) For each L G Choice™ — (S — Optimal™), there exists L' G S — Optimal™ such 
that L -<s L'. Thus, JH, w IF 05 [a ](/9 iff VF G S — Optimal™, Vw' such that w « 
w', Ai,v IF ip for every v G [F]™'. 

Proof (a) (=^>) We assume that © 5 [q;](^ G w. Since ru is a A-MCS, this implies with 
axiom {Cl) that KfE ©5 [a]ip G w , so that the fact that v G ak[w\ yields that 



© 5 [q;]i^ € V. The assumption that Q v then entails that Ka(p G v. (<^=) We 
work by contraposition. Suppose that © 5 [q;](/J ^ w. This implies that © 5 [a]i^ ^ 

V for every v G ak[w\, for if there existed v G a.k[w\ such that (z)s\a\(f G v, 
then Ka'Oi ©5 [a\Lp G v by axiom (Cl) and thus (z)s\oi\F € w, contradicting our 
assumption. Therefore, we take u* G where we know that © 5 [q;](/? ^ u*. 

The union U {Ka'C'ip] KCCxI) G w} U {^Ka^p} is consistent. Let u be the A- 
MCS that includes F)^* U {KaCiip] G w} U {^Kap\. We observe that for 

every v G ak[w], L™ = Then we have that u G ak[w], that = L"* C u, 
and that KaP ^ u. 

(b) Let w' G such that w «« w\ and let v G |Choice“(w)]“'. (^) Assume 
that L™ C w. Taking v in |Choice™(r(;)]™' implies that w v. Let K^p G 

By assumption, we have that K^p G w. Axiom (4) for then implies that 
KaKaP G w. Since w «q- v, K^p G u as well. Therefore, we have shown that 
Fa C u. (<J=) Analogous. 

(c) (=>) Assume that L™ C w. We show that for every L G Choice^, L Choice“(rt;). 
This comes from the fact that for any w' such that such that w k, a w', if Valuer (m* ) = 
1 for some M* G |L]“'n S', then Valuer (o) = 1 for every oG |Choice“(rt;)]™'n 

S. (•<=) We work by contraposition. Assume that L™ % w. The set {m G w; Valuer (u) 
1} is not empty. Take M* G {tt G uJ; Value 5 (M) = 1}. It is the case that Choice™(u*) 
dominates Choice™(ru) in the order -<s. 

(d) Let L G Choice™ — (S — Optimal™). We know that there exists u^, Gw such that 
Value 5 (t 6 *) = 1. From item (c) above, we get that Choice™( m*) G S — Optimal™. 
Notice that the fact that L G Choice™ — (S — Optimal™) implies with items (b) 
and (c) above that that for every w' such that w w', Valuer (o) = 0 for ev¬ 
ery o G |T]™ . Therefore, it is clear that for every w' such that w w' and 
every S G State™', |L]™' n S <s |Choice™(M*)]™' n S (*). On the other 
hand, if we take S* to be the unique state in State™ such that m* G S*, then 
[Choice™(u*)]™ 0 S» {Lja S* (**)■ Our results (*) and (**) together render 
that L ©s Choice™(u*). 

with this we conclude the proof. 

Lemma 4 (Truth Lemma). Let A4 be the canonical Kripke-estit model for A. For 
every formula p of Cko ond every w G W^, we have that M.^w p\S p G w. 

Proof We proceed by induction on p. The cases with the boolean connectives are stan¬ 
dard. It remains to deal with the modal operators. For formulas involving □, [a], and Ka 
both directions follow straightforwardly from Lemma 1 (1), (2), and (3), respectively. 

As for the operators for the objective and subjective oughts, we have the following 
arguments: 

- (“©[«]”) 

(^) We work by contraposition. We assume that ©[a](p f. w. By Lemma 2 (a), 
we get that there exists v' G w such that 27™ C v' and \a\p v'. By Lemma 
1 (2), this implies that there exists v G w such that v'R'fv and such that p v. 

By induction hypothesis, we get that A4,v IF p. From Lemma 2 (c), we know 
that Choice™(u) = Choice™(u') G Optimal™, but this means that there exists 



an “action” in Optimal^ -namely Choice™(u')- such that v € Choice™(u') and 
A^, V (f, which implies by Lemma 2 (d) that A^, w IL ©[a]i^. 

(<J=) We assume that ©[a](p € w. By Lemma 2 (a), we get that [a](p € v' for every 
v' GW such that 27™ C v'. Take L G Optimal™. From Lemma 2 (b) we know that 
27™ C V for every v G L. Therefore, for every v G L, we have that [a\Lp G v, which 
implies by axiom (T) for the stit operator that for every v G L, (p G v. By induction 
hypothesis, we get that for every v G L, Ai,v \\- ip, so that j\4,w IF ©[a](p. 

- (“05 N”) 

(^) We work by contraposition. We assume that © 5 [a](p ^ w. By Lemma 3 (a), 
we get that there exists v' € afc[t«] such that C v' and KaP 4- f^ By Lemma 
1 (3), this in turn implies that there exists v G such that v' v and such that 
p ^ V. Let Oy G tZ; be such that v «« Oy (we know that such Oy exists because 
condition (US)^^ guarantees that the fact that v' G afc[w] implies that there exists 
Oy in w such that v' Oy, so that transitivity of «q, renders that v Oy as 
well). We observe that the fact that Oy v implies that v G [Choice™(ot,)]". 
Now, by induction hypothesis, we get that A4,v p. From Lemma 3 (b) we get 
that the facts that F^ C v' and v' Oy imply that L®” C Oy, so that Lemma 3 
(c) then implies Choice™(o„) G S — Optimal™. This means that there exists an 
“action” in S — Optimal™ -namely Choice™(o„)-such that v G [Choice™(o^)][[ 
and TVf, w IF p, which implies by Lemma 3 (d) that TVf, w IF © 5 [a](p. 

(<^=) We assume that © 5 [a](p G w. By Lemma 3 (a), we get that Kap G v for 
every v G ak[w\ such that F^ C v. Take L G S — Optimal™. From Lemma 3 (b) 
we know that for every w' such that w w' , F^ C v' for every v' G |L]™'. 
Therefore, for every v' G [A]^', we have that KaP G v', which implies -by 
axiom (T) for the knowledge operator- that for every v' G [A]™ , p G v'. By 
induction hypothesis, we get that for every w' such that w w' and v' G [A]™', 
A4,v' IF p. Since A was taken as an arbitrary “action” in S — Optimal™, this 
means that M.,w\\- Qs according to Lemma 3 (d). 

With this we conclude the proof. 

Theorem 1 (Completeness w.r.t. Kripke-estit models). The proof system A is com¬ 
plete with respect to the class of Kripke-estit models. 

Proof Let be a A-consistent formula of Ako- Let F be the A-MCS including p. We 
have seen that the canonical Kripke-estit model is such that p. 

Theorem 2 (Completeness w.r.t. bi-valued BT-models). The proof system A is com¬ 
plete with respect to the class of epistemic utilitarian bi-valued BT-models. 

Proof. Let be a A-consistent formula of Ako- By Theorem 1 we know that there exists 
a Kripke-estit model AA and a world w in its domain such that AA^w IF p. Proposition 
4 then ensures that the epistemic utilitarian bi-valued BT-model associated to AA 
is such that AA"^, {w, hyy) \= p. 



